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This listing of claims will replace all prior versions, and listings, of claims in the application. 
Listing of Claims: 



1 . (currently amended) A method for evaluating improving performance of a 
search mechanism based on context-based user feedback data, said method comprising: 

collecting user information from a user having access to said search mechanism; 
monitoring of said search mechanism for user behavior data regarding an interaction 
of said user with said search mechanism to perform a search; 

monitoring said search mechanism for search mechanism response data regarding said 

search; 

acquiring the context-based user feedback data describing said search by submitting 
one or more questions to the user regarding a performance of the search engine with respect 
to said search and receiving responses to said questions , said questions prompting the user for 
information regarding an extent to which a search result corresponds to a search request ; and 

using the context-based user feedback data to evaluate the performance of identify a 
problem with the search mechanism ; and 

correcting the problem to improve performance of the search mechanism . 

2. (original) The method of claim 1, where said user information comprises one or 
more of the following: 

the speed of said user's connection to said search mechanism; 
the type of said user's connection to said search mechanism; 
a classification of said user's use of said search mechanism; 
background information concerning said user; or 
the language which said user is using to perform said search. 

3. (currently amended) The method of claim 1, where said step of collecting said user 
information comprises: 

requesting said user information from said user; and 
accepting responses from said user. 
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4. (currently amended) At least one of an operating system, a A computer 

readable storage medium having stored thereon a plurality of computer-executable 
instructions , a co processing device, and a computing device for performing the method of 
claim 1. 



5. (currently amended) A method for evaluating improving performance of a 
search mechanism based on context-based user feedback data, said method comprising: 

monitoring of said search mechanism for user behavior data regarding an interaction 
of a user having access to said search mechanism with said search mechanism to perform a 
search; 

monitoring said search mechanism for search mechanism response data regarding said 

search; 

determining if a snooze request specifying a time period to suspend collection of 
explicit feedback data is in effect from said user, and, if not, collecting explicit feedback data 
from said user; 

acquiring context-based user feedback data describing said search, said context-based 
user feedback data comprising information regarding an extent to which a search result 
corresponds to a search request , said context-based user feedback data further comprising 
said explicit feedback data if said explicit user feedback data was collected; and 

using the context-based user feedback data to evaluate the performance of identify a 
problem with the search mechanism ; and 

correcting the problem to improve performance of the search mechanism . 

6. (canceled) 

7. (currently amended) The method of claim 5 [[6]], where said step of determining if a 
snooze request is in effect from said user comprises: 

determining if said user has issued a snooze request; and 

determining if an associated time period associated with said snooze request has 
elapsed. 
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8. (previously presented) The method of claim 5, further comprising: 

storing target data concerning a target value for how often explicit feedback should be 
collected for searches; and 

allowing explicit feedback to be collected only if collecting the explicit feedback 
would not result in exceeding said target value for how often explicit feedback is collected. 

9. (currently amended) At least one of an operating system, a A computer 
readable storage medium having stored thereon a plurality of computer-executable 
instructions , a co processing device, and a computing device for performing the method of 
claim 5. 



10. (currently amended) A method for evaluating improving performance of a 

search mechanism based on context-based user feedback data, said method comprising: 

monitoring of said search mechanism for user behavior data regarding an interaction 
of the user having access to said search mechanism with said search mechanism to perform a 
search, said user behavior data comprising data concerning at least one member of a group 
comprising: requery performed by said user, dwell time on said results page, click time on 
said results page, position of result clicked, more results requested by said user, result dwell 
time, result page size, and result page actions; 

monitoring said search mechanism for search mechanism response data regarding said 

search; 

acquiring context-based user feedback data by submitting one or more questions to 
the user regarding a performance of the search engine with respect to said search and 
receiving responses to said questions context-based user feedback data , said questions 
prompting the user for information regarding an extent to which a search result corresponds 
to a search request ; and 

using the context-based user feedback data to evaluate the performance of identify a 
problem with the search mechanism ; and 

correcting the problem to improve performance of the search mechanism . 

1 1 . (currently amended) A system for evaluating improving performance of a 
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search mechanism based on context-based user feedback data, said system comprising: 

user information collector for collecting user information from a user having access to 
said search mechanism; 

user behavior monitor for monitoring of said search mechanism for user behavior data 
regarding an interaction of said user with said search mechanism to perform a search; 

context monitor for monitoring said search mechanism for search mechanism 
response data regarding said search; and 

context-based user feedback data accumulator for acquiring context-based user 
feedback data describing said search by submitting one or more questions to said user 
regarding a performance of the search engine with respect to said search and receiving 
responses to said questions, said questions prompting said user for information regarding an 
extent to which a search result corresponds to a search request, the context-based user 
feedback data accumulator further for using the context-based user feedback data to evaluate 
the performance of identify a problem with the search mechanism and to improve the 
performance of the search mechanism by correcting the problem . 

12. (original) The system of claim 1 1, where said user information comprises one or 
more of the following: 

the speed of said user's connection to said search mechanism; 
the type of said user's connection to said search mechanism; 
a classification of said user's use of said search mechanism; 
background information concerning said user; or 
the language which said user is using to perform said search. 

13. (original) The system of claim 11, where said user information collector requests 
said user information from said user and accepts responses from said user. 

14. (currently amended) A system for evaluating improving performance of a 
search mechanism based on context-based user feedback data, said system comprising: 

user behavior monitor for monitoring of said search mechanism for user behavior data 
regarding an interaction of a user having access to said search mechanism with said search 
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mechanism to perform a search; 

context monitor for monitoring said search mechanism for search mechanism 
response data regarding said search; 

explicit feedback collection mechanism for making a determination of whether a 
snooze request specifying a time period to suspend collection of explicit feedback data is in 
effect from said user, and, if not, collecting explicit feedback data from said user; and 

context-based user feedback data accumulator for acquiring context-based user 
feedback data describing said search, said context-based user feedback data comprising 
information regarding an extent to which a search result corresponds to a search request , said 
context-based user feedback data further comprising said explicit feedback data if said 
explicit user feedback data was collected, the context-based user feedback data accumulator 
further for using the context-based user feedback data to evaluate the performance of identify 
a problem with the search mechanis m and to improve the performance of the search 
mechanism by correcting the problem . 

15. (canceled) 

16. (currently amended) The system of claim 14 [[5]], where the determination 
of whether a snooze request is in effect from said specific user comprises: 

a determination of whether said user has issued a snooze request; and 
a determination of whether an associated time period associated with said snooze 
request has elapsed. 

17. (previously presented) The system of claim 14, where said explicit feedback 
collection mechanism stores target data concerning a target value for how often explicit 
feedback should be collected for searches; and allows explicit feedback to be collected only if 
collecting the explicit feedback would not result in exceeding said target value for how often 
explicit feedback is collected. 



18. (currently amended) A system for evaluating improving performance of a 

search mechanism based on context-based user feedback data, said system comprising: 
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user behavior monitor for monitoring of said search mechanism for user behavior data 
regarding an interaction of a user having access to said search mechanism with said search 
mechanism to perform a search, said user behavior data comprising data concerning at least 
one member of a group comprising: requery performed by said user, dwell time on said 
results page, click time on said results page, position of result clicked, more results requested 
by said user, result dwell time, result page size, and result page actions; 

context monitor for monitoring said search mechanism for search mechanism 
response data regarding said search; and 

context-based user feedback data accumulator for acquiring context-based user 
feedback data describing said search by submitting one or more questions to said user 
regarding a performance of the search engine with respect to said search and receiving 
responses to said questions, said questions prompting said user for information regarding an 
extent to which a search result corresponds to a search request, the context-based user 
feedback data accumulator further for using the context-based user feedback data to evaluate 
the performance of identify a problem with the search mechanism and to improve the 
performance of the search mechanism by correcting the problem . 
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